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Challenges  in  the  NS  report: 

1.  Dynamics ,  spatial  location,  and  information 
propagation  in  networks. 

2.  Modeling  and  analysis  of  very  large  networks. 

3.  Design  and  synthesis  of  networks. 

4.  Increasing  the  level  of  rigor  and  mathematical 
structure. 

5.  Abstracting  common  concepts  across  fields. 

6.  Better  experiments  and  measurements  of  network 
structure. 

7.  Robustness  and  security  of  networks. 


What  is  “network  science”? 


What  is  science ?  (A  greatly  oversimplified  view.) 

Tentatively  consider  these  subdomains: 

Physics 
Biology 
Social  Sciences 
Math 

Technology 

These  have  enormous  methodological  differences  that 
have  grown  with  time 


What  is  “network  science”? 


(Also  an  oversimplified  but  hopefully  useful  taxonomy) 

Already  exist  distinct  subdomains: 

Network  Technology  (NetTech) 

Network  Biology  (NetBio) 

Network  Social  Sciences  (NetSoc) 

Network  Math  (NetMath) 

Network  Physics  (NetPhys) 

These  have  even  greater  methodological  differences 
than  their  traditional  antecedents 

Spectacular  but  very  uneven  progress 


The  good  news:  Net  Tech 

Network  technology  (interpreted  broadly)  has 
been  wildly  successful. . . 

. . .  yielding  a  “networked  planet”  for  energy, 
food,  information,  goods  and  materials, . . . 


“Network  centric  technology”  has  also  made  it 
possible  to  build  and  demo  almost  anything 
we  can  envision. 


Thus . . . 


Hard,  but  demonstrated  capability 

1.  Dynamics,  spatial  location,  and  information 
propagation  in  networks. 

Modeling  and  analysis  of  very  large  networks. 

Design  and  synthesis  of  networks. 


Needs  focus  and  funding 

•  But  engineers  have  demonstrated  they 
can  solve  such  problems 

7/properly  motivated.  (Big  IF.) 

ooustness  ana  security  oj  networks. 


7. 


The  structure  of  scientific  explanation 


Observe:  Observe  and  formulate  research  plan 
Data:  Gather  and  analyze  data 
Simulate:  Model  and  simulate 
Proof:  Prove  conjectures 

•  Different  branches  of  science  use  these  unevenly 
and  in  very  different  ways. 

•  “Network  science”  is  currently  even  more  extreme. 

•  Network  science  will  demand  a  fundamental 
rethinking  of  their  roles,  particularly  proof. 

•  “Tower  of  Babel”  must  be  avoided. 


Tower  of  Babel? 


1.  Dynamics ,  spatial  location,  and  information 
propagation  in  networks. 

2.  Modeling  and  analysis  of  very  large  networks. 

3.  Design  and  synthesis  of  networks. 

4.  Increasing  the  level  of  rigor  and  mathematical 
structure. 

5.  Abstracting  common  concepts  across  fields. 

6.  Better  experiments  and  measurements  of 
network  structure. 

7.  Robustness  and  security  of  networks. 


The  Net  “Tower  of  Babel” 

NetTech:  global  infrastructure,  Internet,  wireless,  energy, 
transportation,  supply  chain...,  silicon,  fiber,  ...,  network 
centric,  ubiquitous,  embedded-everywhere,  nano-,... 

NetBio:  systems  biology,  genomics,  *omics,  evo-devo, 
bioinformatics,  ...,  metabolic,  transcriptional,  signal 
transduction,  regulatory, . . . ,  medicine,  epidemiology, 
emerging  infections,  medical  disaster  management, 
integrative  physiology,  ecology,  . . . 

NetPhys:  multiscale,  turbulence,  chaos,  stat  mech,...,  fractals, 
criticality,  self-similarity,  scale-free,  cellular  automata, 
universality,  edge-of-chaos,  renormalization,... 

NetSoc:  economics,  sociology,  psychology,  politics, 

management,  law,  ecology...  ,  scaling,  small  worlds .. . 

NetMath:  graph,  information,  control,  dynamical  systems, 
complexity,  formal  methods,  optimization, . . . 


Hard  but  clearly  doable 


Needs  focus  and  funding 
Requires  engagement  of  all  subfields 
Can  be  done 
If  properly  motivated. 


Abstracting  common  concepts  across  fields. 

Better  experiments  and  measurements  of 
network  structure. 

Robustness  and  security  of  networks. 
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The  dominant  challenge 

1.  Dynamics ,  spatial  location,  and  information 
propagation  in  networks. 

2.  Modeling  and  analysis  of  very  large  networks. 

3.  Design  and  synthesis  of  networks. 

4.  Increasing  the  level  of  rigor  and  mathematical 
structure. 

5.  Abstracting  common  concepts  across  fields. 

6.  Better  experiments  and  measurements  of  network 
structure. 

7.  Robustness  and  security  of  networks. 


The  bad  news 


Network  technology  has  been  too  successful. . . 

. . .  yielding  a  “networked  planet”  for  good  and  bad. . . 

. . .  including  all  manner  of  catastrophe  and  destruction. 

“Network  centric  technologies” 

Largely  deliver  what  we  design  them  to  do. 

But  fail  because  they  create  new  problems  we  did  not 
expect. 


The  bad  news 


Network  technology  has  been  too  successful. . . 

. . .  yielding  a  “networked  planet”  for  good  and  bad. . . 
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Robust  yet  fragile 


“Network  centric  technologies” 

Create  extraordinary  capabilities  and  robustness 
And  unexpected  new  fragilities 


Science  deals  poorly  or  not  at  all  with  this  challenge. 
“Network  science”  must  do  better. 


The  dominant  challenge 


Dynamics,  spatial  location,  and  information 
propaga  _ 

We  could  try  our  \,e  networks. 
best  and  still  fail  on 
this  challenge, 
which  would  be 
catastrophic. 
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7.  Robustness  and  security  of  networks. 


Biology:  We  might  accumulate  more  complete 
parts  lists  but  never  “understand”  how  it  all  works. 

Technology:  We  might  build  increasingly  complex 
and  incomprehensible  systems  which  will 
eventually  fail  completely  yet  cryptically. 


Nothing  in  the  orthodox  views  of  complexity 
says  this  won’t  happen  (apparently). 


Interesting  systems  are  robust  yet  fragile. 
Identify  the  fragility,  evaluate  and  protect  it. 
The  rest  (robust)  is  “easy”. 

Nothing  in  the  orthodox  views  of  complexity 
says  this  can’t  happen  (apparently). 


Reason  for  cautious  optimism 


Dynamics,  spatial  location,  and  information 
propagation  in  networks. 

Modeling  and  analysis  of  very  large  networks. 

Design  and  synthesis  of  networks. 

Increasing  the  level  of  rigor  and  mathematical 
structure. 


5.  Abstracting  ca 

6.  Better  experimei 
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7.  Robustness  and  security  of  networks. 


Reason  for  cautious  optimism 


Dynamics,  spatial  location,  and  information 
propagation  in  networks. 

Modeling  and  analysis  of  very  large  networks. 

Design  and  synthesis  of  networks. 

Increasing  the  level  of  rigor  and  mathematical 
structure. 


5.  Abstracting  ca 

6.  Better  experimei 
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7.  Robustness  and  security  of  networks. 


Human  complexity 


Robust 


©  Efficient,  flexible  metabolism 
©  Complex  development 
©  Immune  systems 
©  Regeneration  &  renewal 
|  Complex  societies 
0  Advanced  technologies 


Yet  Fragile 

©  Obesity  and  diabetes 
©  Rich  parasite  ecosystem 
©  Inflammation,  Auto-Im. 
©  Cancer 

&  Epidemics,  war,  . . . 
Catastrophic  failures 


Evolved  mechanisms  for  robustness  allow  for,  even 
facilitate,  novel,  severe  fragilities  elsewhere  (often 
involving  hijacking/exploiting  the  same  mechanism) 

Universal  challenge:  Understand/manage/overcome  this 
robustnes  s/ complexity/ fragility  spiral 
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Remarkably 
common  core 
theoretical 
challenges. 


Yet  also  striking 
increase  in 
unnecessary 
confusion??? 


Core  theory 
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•  Organizational 
principles/structures 

•  Robust  yet  fragile  (RYF) 


Examples:  All 
of  advanced 
technologies, 
all  of  biology 
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catastrophically 

(HOT) 


ims 
iology 


Theoretical  foundations  for  networks 


•  The  most  rigorous,  sophisticated,  and 
applicable  of  existing  theories 

-  Computation 

-  Control 

-  Communications 

•  Have  become  fragmented  and  isolated 

•  Failed  attempts  at  unified  “new  sciences ” 

•  Need  new  mathematics 

•  Recent  progress  has  been  spectacular,  both 
in  rigor  and  relevance 


A  look  back  and  forward 

•  The  Internet  architecture  was  designed  w/o  a  “theory” 

•  Many  academic  theorists  said  it  would  never  work 

•  Recent  “emergent”  theories  are  wildly  wrong  (there  have 
always  been  specious  claims  of  “Achilles  heels”) 

•  We  now  have  a  nascent  and  relevant  theory  that  confirms 
that  the  engineers  were  right  (Kelly,  Low,  Vinnicombe, 
Paganini,  Papachristodoulou,  Li,  Alderson,Willinger,...) 

•  A  strikingly  parallel  (but  less  complete)  story  exists  in 
biology 

•  For  future  networks  and  technologies,  “systems  of 
systems,”  systems  biology  of  the  cell,  organism  and 
brain,  etc,  let’s  hope  we  can  avoid  a  repeat  of  this 
history.  (Looks  like  we  have  a  good  start. . .) 


Background  progress:  Biology 

With  molecular  biology’s  detailed  description  of 
components 

And  growing  attention  to  systems  biology  (e.g.  see 
Stelling,  Covert,  Haddad,  etc) 

The  organizational  principles  of  biological  networks 
are  becoming  increasingly  apparent 

We  are  beginning  to  see  the  principles  of 
architecture  as  well  as  components  and  circuits 

Unfortunately,  we  are  also  seeing  substantial  errors 
and  resulting  confusion  in  the  analysis  of  the 
structure  of  large  networks. 

These  errors  should  be  straightforward  to  correct  but 
have  so  far  been  remarkably  resistant. 


Background  progress:  Technology 


Advanced  technology’s  complexity  is  now 
approaching  biology’s. 

While  the  components  differ,  there  is  striking 
convergence  at  the  network  level  of  architecture  and 
the  role  of  layering,  protocols,  and  feedback  control 
in  structuring  complex  multiscale  modularity. 

For  example,  new  theories  of  the  Internet  and  related 
networking  technologies  have  led  to  test  and 
deployment  of  new  protocols  for  high  performance 
networking. 

The  same  analysis  errors  crept  in  here  but  have 
largely  been  eliminated. 


Background  progress:  Mathematics 


New  mathematical  frameworks  suggests  that  this 
apparent  network-level  evolutionary  convergence 
within/between  biology/technology  is  not  accidental 

But  follows  necessarily  from  universal  requirements 
to  be 

-  efficient, 

-  adaptive, 

-  evolvable, 

-  robust  to  perturbations  in  their  environment  and 
component  parts 


Background  progress:  Mathematics 

This  mathematics  blends  (from  engineering) 
theories  from 

-  optimization, 

-  control, 

-  information,  and 

-  computational  complexity 

with  diverse  elements  in  areas  of  mathematics 
(e.g.  operator  theory  and  algebraic  geometry) 
not  traditionally  thought  of  as  applied. 


Background  progress:  Lessons  learned 


Foundational  issues  this  progress  suggests 

Describe  with  minimal  mathematics  and  minimal 
domain-specific  expertise 

(My  prejudices:  It  must  ultimately  lead  to  a 
mathematical  foundation,  but  it  can’t  start  only  there.) 

Learn  from  success  stories  in  a  broad  sense,  and 
contrast  them  with  failures 


Good  news:  Much  of  the  existing  confusion  can  be 
cleared  up  with  even  minimal  rigor  in  treating  math 
and  data 


Complex  Networks 

Widely  agreed  that  “complexity”  and  “networks”  are 
important  scientific  and  technological  challenges 

Little  broad  agreement  on  even  the  most  basic  notions . . . 

Substantial  (and  unnecessary)  “confusion”  about  even  the 
most  basic,  fundamental,  and  foundational  results  (Tower 
of  Babel) 

Need  a  simpler,  more  coherent,  and  more  rigorous 
taxonomy/ontology  (The  Tower  is  not  unavoidable) 

And  ultimately  a  more  unified,  integrated,  fundamental 
conceptual/theoretical  (i.e.  mathematical)  foundation? 


Confusion  about  basics 


Questions  and  issues: 

•  The  essence  of  “complexity”? 

•  The  foundational  challenges  and  issues? 

•  The  sources  of  so  much  (unnecessary)  confusion? 

•  Existing  success  stories? 

•  Differences  from  the  failures? 

•  Encouraging  new  research? 


Defining  “complexity”:  outline 

•  Aim:  simple  but  universal  taxonomy 

•  Widely  divergent  starting  points  from  math,  biology, 
technology,  physics,  etc, 

•  Can  be  organized  into  a  coherent  and  consistent  picture 

1 .  Review  “simplicity”  in  scientific  exploration 

2.  How  simplicity  breaks  down  and  why 

3.  Distinguish  “emergence”  from  “organization” 

4.  Universal  challenges 

5.  Dramatic  success  stories 

6.  Promising  initial  results 
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Three  key  dimensions  to  complexity  will  be  explored 

•  Small/large  descriptions,  models,  and  theorem  statements 

•  Robust/fragile  features  of  a  system  in  response  to 
perturbations  in  components  or  the  environment. 

•  Short/long  lengths  of  proof  of  properties,  including 
generating  (counter)-examples  by  simulation  (Deeply 
intertwined  with  robust/fragile  dimension.) 


Small 
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Large 
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Robust/ 

Short  proofs 

Simplicity 

Organization 

Fragile/ 

Long  proofs 
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•  Three  key  dimensions  to  complexity  will  be  explored 

•  Four  distinct  kinds  of  complexity  can  be  described  across 
these  three  dimensions. 

•  Robustness/If  agility  and  short/long  are  deeply  connected 
and  their  dimensions  “collapse” 


Much  confusion  comes  from  viewing  this  too  narrowly 


Small 

models 


Fragile/ 
Long  proofs 


Simplicity 


Robust/ 
Short  proofs 


Large 

models 


The  nature  of  simplicity 


Simple  questions: 

•  Small  models 

•  Elegant  theorems 

•  Elegant  experiments 


Simple  answers: 

•  Simple  outcomes 

•  Robust,  predictable 

•  Short  proofs 


Reductionist  science:  Reduce  the  apparent 
complexity  of  the  world  to  an  underlying  simplicity. 

Physics  has  for  centuries  epitomized  the  success  of 
this  approach. 


The  basis  of  rigor 


Simple  questions: 

•  Simple  models 

•  Elegant  theorems 

•  Elegant  experiments 


Simple  answers: 

•  Simple  outcomes 

•  Robust,  predictable 

•  Short  proofs 


•  Mathematical  rigor  is  based  on  theorems  and  proofs 
without  experiments 

•  Physics  “rigor”  is  based  on  elegant  experiments 
with  simple  outcomes  and  no  need  for  theorems, 
proofs,  or  statistics 

•  Complex  systems  research  must  expand  on  these 
sources  of  rigor,  not  retreat  from  rigor  as  an  objective 

•  What  are  challenges  extending  rigor  to  complexity? 


The  basis  of  rigor 

Complex  systems  create  unique  challenges  for  both 
theoretical  and  experimental  rigor,  (particularly  with 
regard  to  architecture): 

-  Scalability  to  large  systems 

-  Robustness  to  component/environment  uncertainty 

-  Evolvability  to  unanticipated  long-term  changes 

Must  explicitly  mix  theory,  simulation,  including 
hardware  and  human  in  the  loop,  and  experiments  in 
new  and  novel  ways 

Huge  issue  needing  careful  thought,  not  enough  said 
here  (sorry) 
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1930s:  The  end  of  certainty 


Simple  questions: 

•  Small  models 

•  Elegant  theorems 

•  Elegant  experiments 


Simple  answers: 

•  Simple  CUtComcs 

•  Pnhiwii  1  '  1,*w 

•  Short  proms 


•  Godel:  Incompleteness,  Turing:  Undecidability 

•  Even  simple  questions  can  be  “complex”  and  fragile 

•  Profoundly  effected  mathematics  and  computation 

•  Modest  impact  on  science,  primarily  through  emphasis 
on  “emergent  complexity” 


“Emergent”  complexity 
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1 960s-Present:  “Emergent  complexity” 


Simple  questions: 

•  Small  models 

•  Elegant  theorems 

•  Elegant  experiments 


Dominates  scientific 
thinking  today 


Unfortunately,  has 
become  a  source  of 
systematic  errors. 


Complexity: 

“New  sciences” 
Unpredictability 
Chaos,  fractals 
Critical  phase  transitions 
Self-similarity 
Universality 
Pattern  formation 
Edge-of-chaos 
Order  for  free 
Self-organized  criticality 
Scale-free  networks 


Small 

Robust/short 

Simplicity 

Fragile/long 

To  make  a 
long  story 
short. . . 


•  Small  applies  to  the  description  of  experiments, 
theorems,  models,  systems 

•  Robust/short  means  experiments  and  real  systems  have 
easily  predictable  behaviors,  and  mathematical  theorems 
have  easy,  short  proofs  or  counterexamples. 

•  Fragile/long  means  unpredictable  results,  difficult  to 
study,  long  proofs,  and  large  changes  in  outputs  due  to 
small  changes  in  inputs  or  models 

•  “Emergence”  means  that  small  (in  all  senses)  can 
nevertheless  lead  to  fragileAong  (also  in  all  senses). 


•  The  most  fragile  details  of  complex  systems  will  likely 
always  be  experimentally  and  computationally  intractable. 

•  Fortunately,  we  care  more  about  understanding,  avoiding, 
and  managing  fragility  (than  about  all  its  details) 
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Long  proofs 
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Organization 
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ribosomes 
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And  now  for  something  completely  different 
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cell  wall 


EGR  control 


Electronic  fuel  injection 


Electronic  ignition 


Anti-lock  brakes 
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Electronic 
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Electric  power 
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1900s:  The  triumph  (and  horror) 


of  organization 
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Flpaont  ditUlLlff? 


Flegard  v^jjciiments 


Simple  answers: 

•  Simple  outcomes 

•  Robust,  predictable 

•  Short  proofs 


•  Complex,  uncertain,  hostile  environments 

•  Unreliable,  uncertain,  changing  components 

•  Limited  testing  and  experimentation 


•  Yet  predictable,  robust,  reliable,  adaptable, 
evolvable  systems 


Organized  complexity 

•  Requires  highly  organized 
interactions,  by  design  or 
evolution 

•  Completely  different  theory 
and  technology  from 
emergence 

•  Complex,  uncertain,  hostile  environments 

•  Unreliable,  uncertain,  changing  components 

•  Limited  testing  and  experimentation 

•  Yet  predictable,  robust,  reliable,  adaptable, 

evolvable  systems 


Simple  answers: 

•  Simple  outcomes 

•  Robust,  predictable 

•  Short  proofs 


Organized  complexity 


Neither  reductionist  science,  nor  (more 
significantly)  “emergent”  complexity  theory  is 
much  help  with  organized  complexity  (and  often 
just  leads  to  further  obfuscation). 

This  no  longer  should  be  a  controversial 
statement,  given  recent  history,  but  it  still  is 

Organized  complexity  has  a  long  history,  but  has 
exploded  in  importance  in  the  last  few  decades 
with  network  technology  and  biology 

What  are  the  keys  to  understanding  organized 
complexity? 


A  look  back  and  forward 

•  The  Internet  architecture  was  designed  w/o  a  “theory” 

•  Many  academic  theorists  said  it  would  never  work 

•  Recent  “emergent”  theories  are  wildly  wrong  (there  have 
always  been  specious  claims  of  “Achilles  heels”) 

•  We  now  have  a  nascent  and  relevant  theory  that  confirms 
that  the  engineers  were  right  (Kelly,  Low,  Vinnicombe, 
Paganini,  Papachristodoulou,  Li,  Alderson,Willinger,...) 

•  A  strikingly  parallel  (but  less  complete)  story  exists  in 
biology 

•  For  future  networks  and  technologies,  “systems  of 
systems,”  systems  biology  of  the  cell,  organism  and 
brain,  etc,  let’s  hope  we  can  avoid  a  repeat  of  this 
history.  (Looks  like  we  have  a  good  start. . .) 


Small 

Large 

Robust 

Simplicity 

Organization 

Fragile 

•  Small  and  Large  apply  to  the  description  of  experiments, 
theorems,  models,  systems 

•  Bio  and  tech  systems  have  enormously  long  and  complex 
descriptions,  yet  extraordinarily  robust  behaviors 

•  Indeed,  robustness  drives  their  complexity,  and  more 
fragile  systems  could  be  much  simpler 

•  Much  of  the  apparent  complexity  of  modem  mathematics 
is  to  create  a  robust  and  rigorous  proof  infrastructure 


Small 

Large 

Robust 

Simplicity 

Organization 

Fragile 

Til 

Even  systems  most  designed/evolved  for  extreme 
robustness,  also  have  fragilities  and  this  is  not  accidental 

The  most  designed/evolved  systems  have  systematic, 
universal  spirals  of  robustness/complexity/fragility 

Understanding/controlling  this  “spiral”  is  arguably  the 
central  challenge  in  organized  complexity 

There  are  hard  constraints  on  robustness/fragility  (from 
theory  not  widely  known  outside  engineering) 

Thus  robustness  is  never  “free”  and  is  paid  for  with 
fragility  somewhere 


Issues 


Small 

Large 

Robust 

Simplicity 

Organization 

Fragile 

•  &»Keige/r>.ce  and  Organization  are  opposites,  but  can  be 

viewed  in  this  unified  framework 

•  &»Keige/r>.ce  celebrates  fragility 

•  Organization  seeks  to  manage  robustness/lf agility 

•  Much  confusion  is  caused  by  failure  to  “get  this” 

•  The  most  fundamental  challenge  of  organizing 
complexity  for  robust  and  evolvable  systems  is  inherent 
and  unavoidable  robustness/fragility  tradeoffs 


Human  complexity 


Robust 


©  Efficient,  flexible  metabolism 
©  Complex  development 
©  Immune  systems 
©  Regeneration  &  renewal 
|  Complex  societies 
0  Advanced  technologies 


Yet  Fragile 

©  Obesity  and  diabetes 
©  Rich  parasite  ecosystem 
©  Inflammation,  Auto-Im. 
©  Cancer 

&  Epidemics,  war,  . . . 
Catastrophic  failures 


Evolved  mechanisms  for  robustness  allow  for,  even 
facilitate,  novel,  severe  fragilities  elsewhere  (often 
involving  hijacking/exploiting  the  same  mechanism) 

Universal  challenge:  Understand/manage/overcome  this 
robustnes  s/ complexity/ fragility  spiral 


Human  complexity  and  RYF 
Robust  Yet  Fragile 

©  Efficient,  flexible  metabolism  ©  Obesity  and  diabetes 
©  Complex  development  and  ©  Rich  parasite  ecosystem 
©  Immune  systems  ©  Inflammation,  Auto-I. 

©  Regeneration  &  renewal  ©  Cancer 
H  Complex  societies  &  Epidemics,  war,  . . . 

ifi  Advanced  technologies  Catastrophic  failures 

•  Inflammation:  classic  example  of  RYF 

-  Normally  robust  and  invisible 

-  Complexity  becomes  apparent  due  to  fragilities 


•  Modem  cars,  planes,  computers,  etc  have 
exploding  internal  complexity 

•  They  are  simpler  to  use  and  more  robust. 

•  They  tend  to  work  perfectly  or  not  at  all. 


Microsoft  Office  PowerPoint  has  encountered  a  problem  and  needs  to  close. 
We  are  sorry  for  the  inconvenience. 


If  you  were  in  the  middle  of  something,  the  information  you  were  working  on  might  be  lost. 


0  Restart  Microsoft  Office  PowerPoint 

Please  tell  Microsoft  about  this  problem. 

We  have  created  an  error  report  that  you  can  send  to  help  us  improve  Microsoft  Office 
PowerPoint.  We  will  treat  this  report  as  confidential  and  anonymous. 


What  data  does  this  error  report  contain? 


Whg  should  I  report  to  Microsoft? 


Microsoft  Office  PowerPoint 


•  Modem  ( 


Send  Error  Report 


Don't  Send 


exploding  internal  complexity 

•  They  are  simpler  to  use  and  more  robust 

•  They  tend  to  work  perfectly  or  not  at  all. 


Microsoft  Office  PowerPoint  has  encountered  a  problem  and  needs  to  close. 
We  are  sorry  for  the  inconvenience. 


If  you  were  in  the  middle  of  something,  the  information  you  were  working  on  might  be  lost. 


0  Restart  Microsoft  Office  PowerPoint 

Please  tell  Microsoft  about  this  problem. 

We  have  created  an  error  report  that  you  can  send  to  help  us  improve  Microsoft  Office 
PowerPoint.  We  will  treat  this  report  as  confidential  and  anonymous. 


What  data  does  this  error  report  contain? 


Whg  should  I  report  to  Microsoft? 


Microsoft  Office  PowerPoint 


Send  Error  Report 


Don't  Send 


Robust  yet  Fragile 


Biology:  We  might  accumulate  more  complete 
parts  lists  but  never  “understand”  how  it  all  works. 

Technology:  We  might  build  increasingly  complex 
and  incomprehensible  systems  which  will 
eventually  fail  completely  yet  cryptically. 


Nothing  in  the  orthodox  views  of  complexity 
says  this  won’t  happen  (apparently). 


Interesting  complex  systems  are  robust  yet 
fragile. 

Identify  the  fragility,  evaluate  and  protect  it. 
The  rest  (robust)  is  “easy”. 


Nothing  in  the  orthodox  views  of  complexity 
says  this  can’t  happen  (apparently). 
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Large 

Robust 

Simplicity 

Organization 

Fragile 

The 

full 

picture 


•  Some  fragilities  are  inevitable  in  robust  complex  systems, 

•  But  are  there  circumstances  in  which  large  descriptions, 
long  proofs,  and  high  fragility  are  desirable ? 

•  Yes,  all  are  important  features  of  cryptography  and 
security,  including  host-pathogen  interactions. 

•  So  organized  complexity  is  not  merely  about  robustness 
but  about  the  management  of  functional  robustness  and 
fragility  (“emergence”  is  avoided  as  much  as  possible) 

•  Emergence  focuses  on  pure  surprise  and  fragility 
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Robust/short 

Simplicity 

Organization 

Fragile/long 

Recap 


Three  key  dimensions  to  complexity 

•  Small/large  descriptions,  models,  and  theorem  statements 

•  Robust/fragile  features  of  a  system/model  in  response  to 
perturbations  in  components  or  the  environment 

•  Short/long  lengths  of  proof  of  properties,  including 
generating  (counter)-examples  by  simulation  (deeply 
intertwined  with  robust/fragile) 


•  Much  existing  confusion  is  created  by  failure  to  grasp 
these  distinctions  (and  lack  of  standard  terminology). 


Robust/ sht^/bra 


Small 


Large 


Organization 


•rreduci 


Recap 


•  The  nightmare  is  that  technology,  biology,  and  medicine 
(and  social  sciences)  get  stuck  with  only  spiraling 
complexity,  large  models/descriptions,  no  coherent 
understanding,  and  uncontrollable  fragilities. 

•  The  hope  is  that  more  rigorous  methods  can  provide 
systematic  tools  for  managing  complexity  and 
robustness/fragility. 

•  There  is  both  tremendous  progress  and  substantial 
confusion,  and  robustness/fragility  is  at  the  heart  of  both. 


Errors 

and 

confusion 
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Organization 

Fragile/Long 

6/)fKeWe/»vee  &  “New  Sciences”  of  Complexity,  Nets,  etc. . . 

•  Confusion  about  origin  and  nature  of  chaos,  power  laws, 
phase  transitions,  fractals,  simulation,  etc 

•  Lack  of  minimal  mathematical  and  statistical  rigor 

•  Misapplication  to  the  organized  complexity  of  biology, 
ecology,  technology  (and  social  systems?) 

•  Classic  (high-impact)  errors  in  ecosystems,  wildfires, 
Internet  topology,  bio  networks,  physiology,  etc 

•  Fortunately,  minimal  impact  in  medicine/technology 


Errors 

and 

confusion 


Small 

Large 

Robust/Short 

Simplicity 

Organization 

Fragile/Long 

Organization  &  the  arrogance  of  technology 

•  Underestimating  the  impact  and  consequences  of 
fragilities  created  by  complexity 

•  Failure  to  manage  complexity/robustness/fragility  spiral 

•  Classic  (high-impact)  consequences  in  global  warming, 
antibiotic  resistance,  software  failures,  spam  and  viruses, 
terrorism  (see  also  “war  against”),  etc. . . 


Errors 

and 

confusion 
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Simplicity 

Organization 

Fragile/Long 

te/jvC-C- 

Organization  and  the  source  of  complexity 

•  Complexity  in  bio  and  tech  networks  is  driven  by  control 
systems  (where/when/how),  not  part  count  (who/what). 

•  Bio:  Protein-coding  gene  count  weakly  correlated  with 
apparent  organized  complexity  of  organism 

•  Tech:  Explosion  in  complexity  in  computer  networks, 
autos,  planes,  devices,  supply  chains,  package  delivery,  etc 
is  in  where/when/how  more  than  who/what. 


Errors 

and 

confusion 
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Simplicity 

Organization 

Fragile/Long 

S/yHeAjfenvC.€- 

IrretfuciMitv) 

Irretjucitle  ccmplexit  ,  intelligent  design,  and  creationism 

•  Overly  mystical  (like  S^-eWe^ee)  and  underestimates  the 
organized  complexity  of  evolved  organisms 

•  Tf  biology  were  irreducibly  complex,  it  would 

-  require  a  (rather  incompetent)  creator,  since  it  would  be 
too  fragile  to  evolve 

-  also  be  so  fragile  as  to  require  constant  intervention  of 
supernatural  control  mechanisms 
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Simplicity 

Organization 

Fragile 

High 

variability? 


•  &»*eWe/|vee  and  Organization  both  involve  high 

variability,  but  with  opposite  mechanisms 

-  S^v-eWeivee  as  the  result  of  bifurcations  to  chaos,  and 

phase  transitions  and  criticality 

-  Organization  as  the  result  of  high  performance 
robust  control  systems 

•  Additional  confusion  is  caused  by  failure  to  “get  this” 
distinction 

•  May  be  very  important  in  acute  illness 


Organization  of  emergent  mechanisms? 


Can  be  exploited, 
But  in  a  managed  way 


Complexity: 

•  “New  sciences” 

•  Unpredictability 

•  Chaos,  fractals 

•  Critical  phase  transitions 

•  Self-similarity 

•  Universality 

•  Pattern  formation 

•  Edge-of-chaos 

•  Order  for  free 

•  Self-organized  criticality 

•  Scale-free  networks 


Robust 


Smp1 


§/yM 


@> 


Robust/fragile  describes  system 
behavior  in  response  to  perturbations 
in  components  or  environment. 


•  Systems  can  be  robust  for  some  features/perturbations  yet 
fragile  in  others 

•  Organized  complexity  involves  systematic  management  of 
robustness/fragility  through  design  or  evolution 

•  There  are  hard  constraints  on  robustness/fragility  (not  to  be 
widely  known  outside  of  engineering  systems) 

•  Theories  on  hard  limits  must  be  driven  by 

-Limits  on  component  technology 
-System-level  requirements 
-Uncertainty  in  environment 


Hard  limits  and  tradeoffs:  Status 


On  systems  and  their  components 

-  Thermodynamics  (Carnot) 

-  Communications  (Shannon) 

-  Control  (Bode) 

-  Computation  (Turing/Godel) 

•  Essential  in  existing  treatments  of  complexity 

•  Fragmented,  incompatible,  incomplete 

•  Need  more  integrated  view  and  have  beginnings 

•  All  have  “robust  yet  fragile”  interpretations 


Robust  design,  Short  proofs,  Small  models 

Key  to  success  of  reductionist  science  in 
study  of  “simple  systems” 

With  largely  different  methods,  also  key  to 
“complex  technology”  success  stories 

-  Internet,  computer,  VLSI 

-  Aerospace,  land,  and  water  vehicles 

-  Optimization-based  scheduling,  manufacturing, 
supply  chain  management,  etc 

-  Etc,  etc... 

What  are  the  essential  issues  and  challenges? 
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Verifiable  robustness  and  evolvability  despite  complex 

systems  with  uncertain  components  in  hostile  environments. 

•  Doesn’t  reductionist  science  already  do  this? 

•  Somewhat,  but  without  reversibility  and  scalability 

•  Reversibility.  Must  provide  systematic  inference  about  the 
full  complex  system,  not  just  its  components 

•  Scalability :  To  large,  highly  organized  systems, 
controlled  far  from  equilibrium,  with  uncertain 
components  and  environments 


To  pursue  the  hope 


This  is  a 

parsimonious  and 
coherent 
ontology  for 
organized 
complexity. 

There  are  many 
“success  stories” 
but  they  are 
fragmented. 


Hard  limits 
Small  models 
Short  proofs 
Robustness/fragility 

Architecture 


Technology  Success  Stories 

Crucial  role  of  soft  ware/ theory  infrastructure 

-  Internet,  computer,  VLSI 

-  Aerospace,  land,  and  water  vehicles 

-  Optimization-based  scheduling,  manufacturing,  supply  chain,  etc 

Small  models 

-  Multiple  scale/resolution/abstraction  models 

-  Robustness  analysis  for  modeling  errors 

-  Restrict  design  space  to  facilitate  computation 

Short  proofs 

-  Simulation  gives  “short  proof’  of  failure 

-  Simulation  cannot  verify  “correctness”  (P^NP^coNP) 

-  Increasing  use  of  formal  verification  and  analysis 


Encouraging  research  progress 

Small  models 

-  System  identification,  machine  learning 

-  Model  reduction  with  error  bounds 

-  More  systematic  multi-scale  modeling 

-  Scalable  models  of  networks  of  interacting  components 
(Internet,  flocking,  coupled  oscillators,...)  with  proofs, 
not  merely  simulations/conjectures 

Short  proofs 

-  Expanded  role  of  convex  optimization  and  duality 

-  Model  checking,  SAT  solvers 

-  SOS  and  P-satz  certificates 

-  Automated  theorem  proving 

-  Performance/robustness  of  distributed/layered  systems 


Encouraging  research  progress 

Hard  limits:  Some  pairwise  unifications 

-  Integrated  Bode-Shannon  theory 

-  Control  theoretic  nonequilibrium  statistical 
mechanics 

-  “Complexity  implies  fragility”  potentially  unifies 
complexity  theory  (P=NP?,  (un)decidability)  with 
numerical  analysis  (condition  number) 


Multiscale  physics  and  models 


Emergent :  Focus  on  most  likely  (and  ignore  rare) 
configurations,  e.g.  equilibrium  statistical 
mechanics,  ensembles,  coarse-graining, 
renormalization 

Organized :  Keep  only  rare  (but  efficient,  robust, 
structured,  evolvable,  etc)  configurations 

Completely  opposite  in  objectives  and  methods 

To  build/evolve  systems  that  push  up  against  hard 
limits  requires  highly  optimized  protocols  and 
modules 

To  build/evolve  systems  that  are  robust  and 
evolvable  requires  architecture 
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systems” 
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Subsystems 


Devices 


“Vertical:” 

connect 

different 

scales 


“Horizontal:”  each  level  is  a 
complex,  often  heterogeneous, 
dynamical  system,  with  rich 
behavior,  and  many  unknowns 


Molecules 


Atoms 
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Traditional  Multiscale  physics 


Condensed 

Molecule 


Atom 


Discard  the 
extremely 
unlikely 


•  Chaos,  fractals 

•  Power  laws 
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Models: 

•  Cellular  au 

•  Boolean  networks 

•  Lattices 
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•  Spin  glasses 

•  Random  graphs 


Critical  phase 
transitions 

Self- similarity 

Universality 

Pattern  formation 

Edge-of-chaos 

Order  for  free 

Self-organized 

criticality 

Scale-free  networks 


Organized  multiscale  technology:  e.g.,  the  Internet 


Enormous  hidden 
complexity 
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Ecosystems 

Cells 


Networks 


Only  the 
extremely 
unlikely 


Genomes 


Molecules 


This  demands  new  ways  of 
describing  multiscale  complexity 


Atoms 


“Architecture”  is  a  central  challenge 


“ The  bacterial  cell  and  the  Internet  have 

-  architectures 

—  that  are  robust  and  evolvable  (yet  fragile? f 

What  does  “architecture”  mean  here? 

What  does  it  mean  for  an  “architecture”  to  be 
robust  and  evolvable? 

Robust  yet  fragile? 

Design  of  architectures  is  replacing  design  of 
systems 


“Architecture”  in  organized  complexity 


•  “Emergence”  may  have  structure  and  (poorly  defined 
notions  like)  “self-organization”  but  no  architecture 

•  Architecture  involves  or  facilitates 

-  System-level  function  (beyond  components) 

-  Organization  and  structure 

-  Protocols  and  modules 

-  Design  or  evolution 

-  Robustness,  evolvability,  scalability 

-  Various  -ilities  (many  of  them) 

-  Perhaps  aesthetics 

•  but  is  more  than  the  sum  of  these 


Hard  limits  and  tradeoffs 


On  systems  and  their  components 

•  Thermodynamics  (Carnot) 

•  Communications  (Shannon) 

•  Control  (Bode) 

•  Computation  (Turing/Godel) 


Assume 
different 
architectures 
a  priori. 


•  Fragmented  and  incompatible 

•  Cannot  be  used  as  a  basis  for 
comparing  architectures 

•  New  unifications  are  encouraging 


Defining  “Architecture” 


The  elements  of  structure  and  organization  that  are 
most  universal,  high-level,  persistent 

Must  facilitate  system  level  functionality 

And  robustness/evolvabality  to  uncertainty  and 
change  in  components,  function,  and  environment 

Architectures  can  be  designed  or  evolve,  but  when 
possible  should  be  planned 

Usually  involves  specification  of  protocols  (rules 
of  interaction)  rather  than  modules 


“ Architecture”  examples 


There  are  universal  architectures  that  are  ubiquitous 
in  complex  technological  and  biological  networks 

Examples  include 

-  Bowties  for  flows  of  materials,  energy,  redox, 
information,  etc  (stoichiometry) 

-  Hourglasses  for  layering  and  distribution  of 
regulation  and  control  (fluxes,  kinetics,  dynamics) 

Nascent  theory  confirms  (reverse  engineers)  success 
stories  but  has  (so  far)  limited  forward  engineering 
applications  (FAST  TCP/AQM) 


Bowties  and  hourglasses 

Bowties  have  large  fan-in  of  diverse  inputs  with 
thin  knots  of  universal  carriers  interfacing  large 
fan-out  of  outputs. 

Example  of  carriers  include: 

-  Packets  in  the  Internet 

-  Lego  snap 

-  Money  in  markets  and  economics 

-  Carriers  and  precursors  in  core  metabolism 

-  Histidine  kinase/Aspartyl  phospho-accepter  in  signal 
transduction  (GPCRs  and  NKkB  in  mammals) 

-  Transcription  and  translation  in  protein  synthesis 

Hourglasses  organize  layered  control  architectures 

-  TCP/IP  in  the  Internet 

-  Control  of  protein  expression  and  protein  degradation 


The  basis  of  rigor  redux 

Complex  systems  create  unique  challenges  in 
creating  both  theoretical  and  experimental  rigor, 
particularly  with  regard  to  architecture : 

-  Scalability  to  large  systems 

-  Robustness  to  component  and  environment  uncertainty 

-  Evolvability  to  unanticipated  changes 

For  example,  we  must  explicitly  mix  theory, 
simulation,  including  hardware  and  human  in 
the  loop ,  and  experiments  in  new  and  novel  ways 
to  investigate  architecture 

Huge  issue  needing  careful  thought,  not  enough 
said  here  (sorry) 
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•  Robustness/fragility  and  short/long  are  deeply 
connected 

•  A  central,  important  conjecture  is  long  proof  complexity 
implies  high  problem  fragility  (equivalently,  robust 
features  have  short  proofs) 

•  There  is  overwhelming  evidence  and  many  special 
cases,  but  no  complete  theory 

•  Important  implication:  Robustness  and  verification  of 
complexity  systems  are  compatible  design  objectives 
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summary 


mi  •  Complex  systems  are  robust  yet  fragile,  with  unavoidable 
constraints  and  tradeoffs 

s  1  •  High  proof  complexity  implies  problem  fragility 
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Thus  robustness  and  its  verifiability  are  compatible 
mm  design  objectives 

llpo[  •  Potentially  good  news  for  the  study  of  all  forms  of 
^  organized  complexity 

Needs  “new  math  and  technology”  not  “new  science” 
Encouraging  beginnings  but  the  math  is  not  yet  readily 
-  accessible  (making  progress  on  both  theory  and  education) 
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